1. Technical Field
The invention is related to image processing in general, and more particularly, to a system and process for generating a composite image from a set of digital images of a static scene captured at different exposure levels. The composite image exhibits an improved uniformity in both exposure and tone in comparison to the set of input images.
2. Background Art
Digital cameras suffer from several deficiencies when compared with traditional photographic film. Among these are a lower dynamic range and a resulting inconsistency in the uniformity of both exposure and tone across an image captured by current digital cameras.
FIGS. 1A through 1C present digital images which will be used to explain the dynamic range problem. FIGS. 1A through 1C show three images of an office desk and window, taken at different exposures. Specifically, these images were captured with a Kodak DCS-40 camera, by adjusting the exposure up and down by two xe2x80x9cstopsxe2x80x9d. FIG. 1B represents an image taken at what was considered the xe2x80x9cproperxe2x80x9d exposure, whereas, the image of FIG. 1A is considered to be xe2x80x9cunderexposedxe2x80x9d and the image of FIG. 1C is considered xe2x80x9coverexposedxe2x80x9d. Notice how the xe2x80x9cunderexposedxe2x80x9d image of FIG. 1A shows better detail for the outside elements (e.g., trees, sky), while the xe2x80x9coverexposedxe2x80x9d image of FIG. 1c shows the interior element (e.g., portraits) better. Also note that while the image of FIG. 1B was taken at what would normally be considered the proper exposure, it still contains areas that appear to be underexposed or overexposed.
The present invention relates to a system and process for manipulating a set of images of a static scene captured at different exposures (i.e., xe2x80x9cbracketedxe2x80x9d images) to yield a composite image with improved uniformity in exposure and tone.
Bracketing is term used in photography meaning to take photographs of the same scene at multiple exposure setting in hopes of obtaining one with the optimum exposure level. This term is borrowed in the context of the present invention to describe a process for overcoming the aforementioned shortcomings of cameras in respect to dynamic range. In general, the invention involves combining multiple images having different exposure levels in such a way as to create a xe2x80x9cbetter lookingxe2x80x9d image. It is noted that for the purposes of this description, the term xe2x80x9cbracketedxe2x80x9d images will mean a set of images having different exposure levels.
A better looking image can be obtained because it has been found that objects depicted in an image of a scene tend to vary in detail depending on the exposure setting. For example, in the previously described experiment it was found that objects in the background of an image appeared in greater detail in an underexposed image, while objects in the foreground showed more detail in an overexposed image. It is believed these inconsistencies resulted in a non-uniformity in the exposure and tone associated with the various pixels making up different parts of the image. The goal of the subject invention is to produce a single image from images of a static scene taken at different exposures, which captures to some extent, the portions of each image providing the best exposure and tone characteristics.
In general, the aforementioned goal can be achieved by analyzing a set of bracketed images using a multi-dimensional histogram and merging the images via an approach that projects pixels onto a curve that fits the data. However, it has been found that the desired composite image having improved exposure and tone can also be produced in a simpler manner by summing characteristic pixel values across the multiple images, followed by an equalization process. The characteristic pixel values are preferably any appropriate measure of the pixel brightness level (also known as the luminous intensity value) exhibited by a pixel. For example, if the images are black and white, the pixel gray level could be used. If color images are involved, the Y-luminance channel could be used.
One possible equalization process would be to simply average the chosen characteristic pixel values. This is accomplished by dividing the summed value of each pixel set by the number of images in the set of images. It is noted that a pixel set is made up of all the corresponding pixels from the bracketed images, where corresponding pixels are defined as the pixels that represent the same portion of the depicted scene. While this simple averaging method does produce a better looking picture, an even better apparent dynamic range can be achieved using a histogram equalization process.
In essence, this histogram equalization involves creating a count of the number of pixels sets having the same summed brightness level. From this count, a cumulative distribution function is computed and normalized to a maximum value corresponding to the maximum summed brightness level. The cumulative distribution function is then used to determine new pixel brightness levels to use in generating the composite image. Specifically, the normalized cumulative count value that corresponds to the summed brightness value associated with each pixel set is identified and divided by the number of images in the set of bracketed images to produce a new pixel brightness value associated with each pixel set. The composite image is generated by using the new pixel brightness value associated with a pixel set as the pixel value for a pixel in the composite image corresponding to the location of the pixels of the associated pixel set in the scene depicted by the images.
A further refinement can be implemented by employing a partial equalization approach. Partial equalization can be described as the blending of the aforementioned normalized cumulative distribution function with a straight line function. The degree of equalization of the image is controlled by the percentage of blending of the straight line function with the cumulative distribution function (i.e. any blend between 0% and 100% is possible).
In addition to the just described benefits, other advantages of the present invention will become apparent from the detailed description which follows hereinafter when taken in conjunction with the drawing figures which accompany it.